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1. INTRODUCTION 

The use of vehicles in our daily lives is increasing dramatically, and as the number of vehicles grows, 
more and more of them are breaking traffic laws. Robberies of automobiles, entry into restricted areas, and an 
unusually high number of accidents all contribute to an increase in the number of crimes. Throughout the globe, 
a vehicle registration number will be critical in identifying any vehicle and ensuring that it is properly 
identified. It primarily serves to strengthen the overall safety and security of the system. In addition to being 
employed at entrance points to different workplaces, societies, and other establishments, the vehicle license 
plate (LP) recognition system may also be used for vehicle authentication [1], [2]. This technique has the 
potential to be completely automated. The access technique for visitors, personnel, or understudy visiting the 
region includes a security labor force that validates the facts by reviewing the identifiable proof archive. This 
is true for the majority of the vehicle entry ports. Due to the fact that this is a manual process, there is a 
significant likelihood that human errors may occur throughout the affirmation and enrolment procedures. 
Employees assigned to the specified location may find the work of manually recording and compiling material 
to be time-consuming and difficult to do successfully. 
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The majority of car license plate recognition systems are designed on traffic images or video analysis, 
in which case computer vision methods are often used to recognizelicense plates. The use of machine learning 
and deep learning algorithms, which display improved classification accuracy and resilience, has been a 
significant recent breakthrough. Several car-related applications, including vehicle identification, vehicle 
categorization, vehicle plate recognition, and road condition monitoring, have benefited from the development 
of deep learning and computer vision approaches [3]. Deep learning approaches, as opposed to computer vision 
methods, offer certain benefits in terms of generalization capacity and resistance to uncertainties, noise, and 
occlusion in images, but at the expense of a larger computing burden and a greater need for sample set size. In 
order to take advantage of these newly developed techniques, several methods have been proposed, including 
the dirty number plate detection system based on you only look once (YOLO) [4], [5] the license number plate 
recognition system based on convolutional neural network (CNN) [6], recurrent neural network (RNN) [7], 
and support vector machines (SVM) [8]. 

Tayara et al. [9] suggests thatfew drawbacks exist, however, when it comes to using deep learning 
algorithms for identifying vehicles and their license plates. Complex environments surrounding the car, 
including buildings, trees, people, and other things, cause deep neural networks to get confused and hence 
impair the accuracy of detection. In addition, the human aspect, the vehicle's speed, and the driver's conduct 
are taken into consideration. All of the factors listed above make achieving precise and efficient detection more 
challenging. This paper presents an automated authentication technique that would eliminate the need for 
human verification and decrease the amount of labor required of security personnel. The rest of the paper is 
organized as follows: the remaining part of section 1 gives the related works, section 2 explains the proposed 
method, section 3 presents the experiments and analysis, and section 4 concludes the paper. 

A vehicle's license plate serves as an identification number. So, in an intelligent driving system, the 
automated detection and identification of license plates is one of the most significant responsibilities. In order 
to recognizelicense plates, a variety of techniques have been explored. Kumar et al. [10] suggested that the 
histogram method, which was only tested by cars from five nations, was used to determine the range of a 
license plate using the back vehicle lights. YOLO was suggested for automated license plate identification [4], 
which had a recognition accuracy of 78%. Silva and Jung [11] presented a method to simultaneously identify 
and recognize all characters in a cropped image. YOLO's first eleven layers and four additional convolutional 
layers are combined to enhance non-linearity. Later YOLOv2 model is developed in [12], while in [13], the 
spatial pyramid pooling was added to YOLOv3. They have good identification rates in different datasets, but 
its LP recognition depth makes it impossible to match the real-time needs of vehicle license detection system 
(VLDS) applications, notwithstanding their success. A bi-directional long short-term memory (LSTM) network 
was used in [14] to find the characters on the LP. A 1-D attention module was investigated to extract important 
information from the character areas in order to improve the accuracy of LP identification. Zhang et al. [15] 
used a 2-D attention mechanism to improve their recognition model, which relies on the feature extraction 
capabilities of an Xception-based 30-layer CNN. 

Wang et al. [16] was presenteda weight-sharing classifier that can detect all occurrences of a character 
in any place. Small cross-dataset studies have been conducted in recent years to test the generalizability of the 
suggested approaches, since recognition rates using the classic split methodology have grown dramatically in 
recent years. The images from chinese city parking dataset (CCPD) dataset is used to train their recognition 
models for Chinese LPs. Silvano et al. [17] developed a system that combines Mercosur LPs with real-world 
images with different LP layouts. The findings of a model trained only on synthetic images were encouraging, 
but it is impossible to evaluate these results correctly since the test images were not made accessible to the 
scientific community. 

LP of motorbikes frequently contain two rows of characters, which are difficult for 
sequential/recurrent-based algorithms, and since they are smaller in size and sometimes slanted, they have been 
removed from several studies. In order to give LPs with one or two rows of characters the same weight in the 
evaluation of VLDS systems, there is a high need for a public dataset for end-to-end VLDS with the same 
number of images of vehicles and motorbikes. However, real-world applications still have a problem with the 
accuracy of the rate. In addition, most of these technologies demand a substantial amount of computing power 
since they require the system to search through vast and high-resolution images or videos that were taken from 
the road. 

Deep learning is a machine learning approach based on neural networks [18]-[21]. Various deep 
learning network structures, including as AlexNet, over feat, GoogLeNet, and ResNet, have been depleted. 
Because of its tremendous performance in image processing, region-based convolutional neural network 
(RCNN) is the most used deep learning model. The RCNN is built up of neurons with weights and bias that 
may be adjusted. Images may be used as input data, and the layers of the RCNN contain three dimensions 
(width, height, and depth). RCNNs are commonly utilised in the detection of vehicles and vehicle number 
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plates. In order to improve the recognition accuracy, in this study a multi layered feed forward neural network 
will be used to recognise automobile license plates with high accuracy and speed. 


2. PROPOSED METHOD 

Regular monitoring of cars in a hectic traffic environment necessitates a complicated system. 
Detecting cars with adequate accuracy is crucial to the overall system functioning. Figure 1 shows the overall 
workflow of the proposed system. In this paper, character level number plate recognition is accomplished using 
the multi layered feed forward back propagation neural network (MLFFBPNN). The moving vehicles are 
identified in the vehicle level detection. The plate level detection detects the number plate area once the vehicle 
has been located. Instead of seeing the whole image, this level concentrates on the license plate zone. This 
significantly minimises the computation complexity. Following that, MLFFBPNN is used to perform character 
level recognition. 


Vehicle Identification 


License Plate Recognition 


Character Level License Plate Recognition 


Figure 1. Workflow of the proposed system 


The character level number pate recognition using MLFFBPNN has been described in Figure 2. In 
this method, there are 3 layers namely input, hidden, and output layers has been utilized. Each layer has been 
related with interconnection weights. In feed forward of information, initially a set of randomly chosen weights 
are feed to the input data and an output has been determined. Back propagation training algorithm is utilized 
to train the network. During back propagation, the propagation error has been determined, then the new weights 
are calculated and the weights are attuned for the minimization of error. This process has been repeated until 
the weights are calibrated to exactly predict an output [22]. 


Multiple Layers 


Output Layer 


- EF 


Figure 2. MLFFBP neural network 
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2.1. Number of neurons in the hidden layer 

Rule-of-thumb method been utilized to determine the number of neurons in the hidden layer. Let Ni 
be the number of input neurons, Nh be the number of hidden neurons and No be the number of output neurons 
[23], [24]. 10 input neurons and 15 hidden neurons have been used in this network. Gray level co-occurrence 
matrix (GLCM) properties have been used as an input feature. 


2.2. Back propagation training algorithm 
Backpropagation is a widely used algorithm for training feedforward neural networks. The steps 
involved in algorithm are as follows: 

a. Initialization of weights-small random values are selected as initial weights. 

b. Feed forward- all the input features are transmitted to the hidden layer through input layer. The sigmoid 
function has been determined in both the hidden layer and the output layer. The required consequences are 
obtained from the output layer. 

c. Back propagation of errors-the associated error has been determined by comparing the activation y,’ and 
the target value yo. Then the new weights are computed for error minimization. 

d. Updation of the weights. 


2.3. Estimation of change in weight 
The total error can be represented as, 


-iF n2 
e=3 (Yo — Yo) 


The relationship among the weight and the error can be written as, 


w* de 
x — — 
ow 
è a 
Wo X = 55 — (1) 
It can be rewritten as, 
eet OE AVG OWneto 
Who Ta 3yo 3Wneto Wmo (2) 
Consider the terms separately and simplify, 
ae _ A(300-Yo)? ; 
D7 G F l= -o =y) (3) 
ay, _ a(1+e-Wneto) * _ e-Wneto ial cea! 
Wnes Wreg Gre- Wneto)? = ¥o(1 — Yo) (4) 
OWneto _ 3WmoYm nl 
3Wmo Wmo YM (5) 
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Where, 5, = (Yo — Yo)Yo (1 — yo). To find the Weight change for an input layer to hidden layer, 
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3. METHOD 
3.1. Dataset 

The Universaldade Federal do parana-automated license plate readers (UFPR-ALPR) dataset contains 
4,500 fully annotated images taken from 150 cars in real-world circumstances in which both the vehicle and 
the camera (located inside another vehicle) are moving. The images were taken using three separate cameras 
and are in the portable network graphics (PNG) format, with a resolution of 1,920 by 1,080 pixels. The dataset 
is divided into three parts: 40% for training, 40% for testing, and 20% for validation. Figure 3 shows some of 
the UFPR-ALPR dataset's sample images [25]. 


Figure 3. Test images of UFPR-ALPR dataset 


3.2. RodoSol-ALPR 

The Rodovia do Sol (RodoSol)-ALPR dataset contains 20,000 images collected by pay toll cameras 
operated by the RodoSol concessionaire, which runs 67.5 kilometres of highway (ES-060) in the Brazilian state 
of Esprito Santo. Images of various sorts of vehicles (e.g., automobiles, motorbikes, buses, and trucks) were 
taken at different times of day and night, from different lanes, on clear and rainy days, and the distance between 
the vehicle and the camera fluctuates somewhat. All of the images have a 1,280 720 pixel resolution. Figure 4 
shows some of the UFPR-ALPR dataset's sample images. 


Figure 4. Test images of RodoSol-ALPR dataset 
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3.3. Simulation results of character level recognition 

The RodoSol-ALPR and the UFPR-ALPR dataset is used to train the MFFBPNN. The vehicle level 
and plate level recognition of images are given in Figure 5. In this work, we have considered 500 epochs. The 
best performance is obtained with the mean squared error of 0.16529, this has been given in Figure 6. 
Figure 7 shows training, validation and testing performance. 


Figure 5. Vehicle level and plate level recognition 


Best Training Performance is 0.16529 at epoch 0 


Mean Squared Error (mse) 


o 50 100 150 200 250 300 350 400 450 500 
500 Epochs 


Figure 6. Mean squared error 
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Figure 7. Training, validation and testing performance 


4. RESULTS AND DISCUSSION 

In the 12-layer MLFFBPNN (with two convolution layers) and the original (3 convolution layer) 
networks, the weight of the first convolution layer is adjusted once training is completed. The weights of the 
networks reveal that both networks have a good grasp of the dataset's visuals. Therefore, the structural 
adjustment does not have a substantial impact on the ability to extract features. In the next experiment, the 
middle layers are further reduced to a single convolution layer (9-layer RCNN). The image characteristics were 
not captured by the weights of the single convolution network. Two convolution layers have been removed, 
reducing the learning ability. The accuracy of the 12-layer and 15-layer MLFFBPNNs during training iterations 
is almost identical. Learning capacity is unaffected by a decrease in the number of layers. The 9-layer 
MLFFBPNN, on the other hand, has a much lower accuracy rate of 32.3%. Figure 8 (a) to (c) shows the results 
of the suggested technique, as presented in the Appendix. 

Table 1 gives the accuracy of the four MFFBPNNs after the intermediate layers were trained. The 
accuracy is calculated by dividing each letter’s recognition sample by the overall sample value and multiplying 
by 100. The comparison of recognition accuracy of MLFFBPNN and RCNN is given in Table 2. Although the 
classification capacity is reduced when a convolution layer is removed from the MLFFBPNN in vehicle and 
plate levels have similar accuracy. This is because the plate detection issue is simpler than the vehicle 
identification problem, and hence a smaller MLFFBPNN can achieve the same accuracy. This also implies that 
the all-in-one MLFFBPNN may be tweaked without significantly affecting its accuracy. Because characters 
have fewer characteristics, their degree of accuracy is lower than that of plates or vehicles. As a result, the 
character-classification issue becomes a multiclass issue. Finally, extra training sets should be considered for 
the character's final layers. Experiments showed that the suggested technique worked by reducing MLFFBPNN 
and improving overall system performance. The comparative analysis of recognition accuracy of MLFFBPNN 
and the RCNN is given in Figure 9. From the analysis, it is clearly show that the proposed method gives better 
accuracy in character level recognition. 


Recognition Sample 


Accuracy(%) = ( ) * 100 


Total Sample 


Table 1. Comparative analysis of accuracy and computation time 


Method Layers Training Time (min) _ Accuracy 
RCNN 15 49 56.7 
12 44 52.5 
9 32 10 
MLFFBPNN 15 32 62.4 
12 30 61.9 
9 27 32.3 
3 21 17.2 
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Table 2. Comparison of recognition accuracy 


Methods Vehicle level Plate level Character level 
MLFFBPNN 98 98 89 
RCNN 98.5 97 85 
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Figure 9. Recognition accuracy of MLFFBPNN and RCNN 


5. CONCLUSION 


In this paper, deep learning based MLFFBPNN has been presented for vehicle license plate detection 
and recognition. Here, three levels of recognition have been performed such as vehicle level, plate level, and 
character eve recognition. The proposed work is executed with the different number of layers. Here, we 
observed that, when the number of layers increased, the training time and the accuracy also get increased. We 
get the better performance with 15 layers. The performance of MLFFBPNN is compared with the RCNN 
network. The recognition accuracy of the proposed method is 89% and it is improved by 4% when compared 
with RCNN. In future, we planned to design a separate network for each level based on particular properties 
in the respective subtask and enhance MLFFBPNN's character level to enhance plate recognition. 
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Figure 8. Character level LP recognition (a) plate level detection, (b) manual detection, and 
(c) character level detection 
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Figure 8. Character level LP recognition (a) plate level detection, (b) manual detection, and (c) character 
level detection (continue) 
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